Analysis of cardiac performance using ultrasonic diagnostic images

ABSTRACT

Ultrasonic cardiac image information is acquired and segmented by automatic border detection. The segmented ultrasonic information is used to display regional wall motion over time. The segmented information may be presented in a color-coded representation, or entered automatically as qualitative or quantitative measures on a scorecard of cardiac performance. The inventive technique is applicable to both two dimensional and three dimensional ultrasonic image information.

This invention relates to ultrasonic diagnostic imaging systems and, inparticular, to ultrasonic diagnostic imaging systems which automaticallyassist in the assessment of cardiac performance and deficiencies.

Many ultrasonic diagnostic procedures in which bodily functions andstructures are quantified rely upon clear delineation and definition ofthe body structures and organs which are being measured. When thequantification or measurement procedure uses static images or a smallset of measurements, the delineation of the bodily structure beingmeasured can be done manually. An example of such a procedure is theobstetrical measurements of a developing fetus. Static images of thedeveloping fetus can be acquired during periods when fetal activity islow. Once an image is acquired, only a few circumference or lengthmeasurements are usually required to compute development characteristicssuch as gestational age and anticipated delivery date. Thesemeasurements can readily be made manually on the fetal images. Otherdiagnostic procedures, particularly those involving measurements of theheart and its functioning, present a further set of difficulties. Theheart is always beating and hence is always in motion. As it moves, thecontours of the heart constantly move and change as the organ contractsand expands. To fully assess many characteristics of cardiac function itis necessary to evaluate many and at times all of the images acquiredduring the heart cycle (one heartbeat), which can amount to thirty toone hundred and fifty or more images. The structure of interest such asthe endocardium, epicardium or valves must then be delineated in each ofthese images, a painstaking, time-consuming task. Since these structuresare constantly in motion, they appear slightly different in each imageacquired during the cardiac cycle, and can also vary significantly fromone patient to another. While applications such as obstetricalprocedures would benefit from a processor which automatically delineatesspecific anatomy in an ultrasonic image, cardiac diagnosis would benefiteven more so.

Once anatomy can automatically be delineated in cardiac images, it thenbecomes desirable how to make the best diagnostic use of the delineatedimage information. It is desirable, for instance, to convey a sense ofregional wall motion but in a format which enables the user tospecifically diagnose particular locations of the heart wall and to beable to do so over the full cardiac cycle. Furthermore, it is desirableto give the user a visual, automated assessment of regional cardiacperformance, enabling the user to quickly pinpoint and thereafterexamine potentially deficient regions of the heart.

In accordance with the principles of the present invention, ultrasoniccardiac image information is acquired and segmented by automatic borderdetection. The segmented ultrasonic information is used to displayregional wall motion over time. The segmented information may bepresented in a color-coded representation, or entered automatically asqualitative or quantitative measures on a scorecard of cardiacperformance. The inventive technique is applicable to both twodimensional and three dimensional ultrasonic image information.

In the drawings:

FIG. 1 is a four chamber ultrasound image of the heart;

FIG. 2 illustrates an ultrasound display of both end diastole and endsystole cardiac images;

FIGS. 3a and 3 b illustrate the step of locating the medial mitralannulus (MMA) and the lateral mitral annulus (LMA) in an ultrasoundimage of the left ventricle (LV);

FIG. 4 illustrates the step of locating the apex of the LV;

FIGS. 5a-5 c illustrate standard border shapes for the LV;

FIGS. 6a-6 b illustrate geometric templates used to locate the MMA andLMA;

FIGS. 7a-7 c illustrate a technique for fitting a standard border shapeto the endocardial boundary of the LV;

FIG. 8 illustrates an end diastole and end systole display withendocardial borders drawn automatically in accordance with theprinciples of the present invention;

FIG. 9 illustrates the rubber-banding technique for adjusting anautomatically drawn border;

FIG. 10 illustrates the selection of a cardiac cycle by viewingautomatically drawn borders;

FIG. 11 illustrates a tissue Doppler map of endocardial motion over aplurality of cardiac cycles;

FIGS. 12 illustrates the use of automated border detection to segment animage of the heart wall;

FIGS. 13a and 13 b illustrate scorecards for scoring segments of theheart wall;

FIGS. 14a and 14 b illustrate techniques for making strain ratemeasurements as a function of tissue motion;

FIGS. 15a-15 c illustrate 3D techniques for evaluating cardiacperformance;

FIG. 15d illustrates a scorecard for scoring a three dimensional imageof the heart; and

FIG. 16 illustrates in block diagram form an ultrasonic diagnosticimaging system constructed in accordance with the principles of thepresent invention.

Referring first to FIG. 1, an ultrasound system display is shown duringthe acquisition of cardiac images. The ultrasound image 10 is afour-chamber view of the heart which is acquired by a phased arraytransducer probe to produce the illustrated sector-shaped image. Theimage shown is one of a sequence of real-time images acquired byplacement of the probe for an apical 4-chamber view of the heart, inwhich the probe is oriented to view the heart from the proximity of itsapex 11. The largest chamber in the image, in the central and upperright portion of the image, is the left ventricle (LV). As the real-timeultrasound image sequence is acquired an ECG trace 12 of the heart cycleis simultaneously acquired and displayed at the bottom of the display,with a triangular marker 14 denoting the point or phase of the cardiaccycle at which the currently-displayed image was acquired. A typicalduration of the heart cycle when the body is at rest is about onesecond, during which time approximately 30-90 image frames of the heartare acquired and displayed in rapid succession. A sequence of imageframes for a heart cycle is referred to herein as a “loop” of images, asa clinician will often acquire and save the sequence of images of aheart cycle and then replay them in a continuous “loop” whichrepetitively displays the selected cardiac cycle. As the clinician viewsthe display of FIG. 1, the heart is seen beating in real time in theultrasound display as the ECG waveform 12 scrolls beneath the ultrasoundimages 10, with the instantaneously displayed heart phase indicated bythe marker 14.

In one mode of acquisition in accordance with the present invention, theclinician observes the beating heart in real time while manipulating thetransducer probe so that the LV is being viewed distinctly in maximalcross-section. When the four chamber view is being acquired continuouslyand clearly, the clinician depresses the “freeze” button to retain theimages of the current heart cycle in the image frame or Cineloop® memoryof the ultrasound system. The Cineloop memory will retain all of theimages in the memory at the time the freeze button is depressed which,depending upon the size of the memory, may include the loop being viewedat the time the button was depressed as well as images of a previous orsubsequent loop. A typical Cineloop memory may hold 400 image frames, orimages from about eight to ten heart cycles. The clinician can then scanthrough the stored images with a trackball, arrow key, or similarcontrol to select the loop with the images best suited for analysis.When the clinician settles on a particular loop, the “ABD” protocol isactuated to start the border drawing process.

When the ABD protocol is actuated the display changes to a dual displayof the end diastole image 16 and the end systole image 18 displayedside-by-side as shown in FIG. 2. The ultrasound system identifies all ofthe images comprising the selected loop by the duration of the ECGwaveform associated with the selected loop. The ultrasound system alsorecognizes the end diastole and end systole points of the cardiac cyclein relation to the R-wave of the ECG waveform 12 and thus uses the ECGwaveform R-wave to identify and display the ultrasound images at thesetwo phases of the heart cycle. The dual display of FIG. 2 shows the ECGwaveform 12 for the selected heart cycle beneath each ultrasound image,with the marker 14 indicating the end diastole and end systole phases atwhich the two displayed images were acquired.

Since the Cineloop memory retains all of the images of the cardiaccycle, the user has the option to review all of the images in the loop,including those preceding and succeeding those shown in the dualdisplay. For instance, the clinician can “click” on either of the imagesto select it, then can manipulate the trackball or other control tosequentially review the images which precede or succeed the one selectedby the ultrasound system. Thus, the clinician can select an earlier orlater end diastole or end systole image from those selected by theultrasound system. When the clinician is satisfied with the displayedimages 16 and 18, the ABD processor is actuated to automaticallydelineate the LV borders on the two displayed images as well as theintervening undisplayed images between end diastole and end systole.

In this example the ABD processor begins by drawing the endocardialborder of the LV in the end systole image 18. The first step in drawingthe border of the LV is to locate three key landmarks in the image, themedial mitral annulus (MMA), the lateral mitral annulus (LMA), and theendocardial apex. This process begins by defining a search area for theMMA as shown in FIG. 3a, in which the ultrasound image grayscale isreversed from white to black for ease of illustration. Since the ABDprocessor is preconditioned in this example to analyze four-chamberviews of the heart with the transducer 20 viewing the heart from itsapex, the processor expects the brightest vertical nearfield structurein the center of the image to be the septum which separates the left andright ventricles. This means that the column of pixels in the image withthe greatest total brightness value should define the septum. With thesecues the ABD processor locates the septum 22, and then defines an areain which the MMA should be identified. This area is defined fromempirical knowledge of the approximate depth of the mitral valve fromthe transducer in an apical view of the heart. A search area such asthat enclosed by the box 24 in FIG. 3a is defined in this manner.

A filter template defining the anticipated shape of the MMA is thencross correlated to the pixels in the MMA search area. While thistemplate may be created from expert knowledge of the appearance of theMMA in other four-chamber images as used by Wilson et al. in their paper“Automated analysis of echocardiographic apical 4-chamber images,” Proc.of SPIE, August, 2000, the present inventors prefer to use a geometriccorner template. While a right-angle corner template may be employed, ina constructed embodiment the present inventors use an octagon cornertemplate 28 (the lower left corner of an octagon) as their searchtemplate for the MMA, as shown at the right side of FIG. 6a. Inpractice, the octagon template is represented by the binary matrix shownat the left side of FIG. 6a. The ABD processor performs templatematching by cross correlating different sizes of this template with thepixel data in different translations and rotations until a maximumcorrelation coefficient above a predetermined threshold is found. Tospeed up the correlation process, the template matching may initially beperformed on a reduced resolution form of the image, which highlightsmajor structures and may be produced by decimating the original imageresolution. When an initial match of the template is found, theresolution may be progressively restored to its original quality and thelocation of the MMA progressively refined by template matching at eachresolution level.

Once the MMA has been located a similar search is made for the locationof the LMA, as shown in FIG. 3b. The small box 26 marks the locationestablished for the MMA in the image 18, and a search area to the rightof the MMA is defined as indicated by the box 34. A right cornergeometric template, preferably a right octagon corner template 38 asshown in FIG. 6b, is matched by cross-correlation to the pixel values inthe search area of box 34. Again, the image resolution may be decimatedto speed 35 the computational process and different template sizes maybe used. The maximal correlation coefficient exceeding a predeterminedthreshold defines the location of the LMA.

With the MMA 26 and the LMA 36 found, the next step in the process is todetermine the position of the endocardial apex, which may be determinedas shown in FIG. 4. The pixel values of the upper half of the septum 22are analyzed to identify the nominal angle of the upper half of theseptum, as indicated by the broken line 43. The pixel values of thelateral wall 42 of the LV are analyzed to identify the nominal angle ofthe upper half of the lateral wall 42, as shown by the broken line 45.If the lateral wall angle cannot be found with confidence, the angle ofthe scanlines on the right side of the sector is used. The angle betweenthe broken lines 43,45 is bisected by a line 48, and the apex isinitially assumed to be located at some point on this line. With thehorizontal coordinate of the apex defined by line 48, a search is madeof the slope of pixel intensity changes along the line 48 to determinethe vertical coordinate of the apex. This search is made over a portionof line 48 which is at least a minimum depth and not greater than amaximum depth from the transducer probe, approximately the upperone-quarter of the length of line 48 above the mitral valve planebetween the MMA 26 and the LMA 36. Lines of pixels along the line 48 andparallel thereto are examined to find the maximum positive brightnessgradient from the LV chamber (where there are substantially no specularreflectors) to the heart wall (where many reflectors are located). Apreferred technique for finding this gradient is illustrated in FIG. 7.FIG. 7a shows a portion of an ultrasound image including a section ofthe heart wall 50 represented by the brighter pixels in the image. Drawnnormal to the heart wall 50 is a line 48 which, from right to left,extends from the chamber of the LV into and through the heart wall 50.If the pixel values along line 48 are plotted graphically, they wouldappear as shown by curve 52 in FIG. 7b, in which brighter pixels havegreater pixel values. The location of the endocardium is not the peak ofthe curve 52, which is in the vicinity of the center of the heart wall,but relates to the sense of the slope of the curve. The slope of thecurve 52 is therefore analyzed by computing the differential of thecurve 52 as shown by the curve 58 in FIG. 7c. This differential curvehas a peak 56 which is the maximal negative slope at the outside of theheart wall (the epicardium). The peak 54, which is the first major peakencountered when proceeding from right to left along curve 58, is themaximal positive slope which is the approximate location of theendocardium. The pixels along and parallel to line 48 in FIG. 4 areanalyzed in this manner to find the endocardial wall and hence thelocation of the endocardial apex, marked by the small box 46 in FIG. 4.

Once these three major landmarks of the LV have been located, one of anumber of predetermined standard shapes for the LV is fitted to thethree landmarks and the endocardial wall. Three such standard shapes areshown in FIGS. 5a, 5 b, and 5 c. The first shape, border 62, is seen tobe relatively tall and curved to the left. The second shape, border 64,is seen to be relatively short and rounded. The third shape, border 66,is more triangular. Each of these standard shapes is scaledappropriately to fit the three landmarks 26, 36, 46. After anappropriately scaled standard shape is fit to the three landmarks, ananalysis is made of the degree to which the shape fits the border in theecho data. This may be done, for example, by measuring the distancesbetween the shape and the heart wall at points along the shape. Suchmeasurements are made along paths orthogonal to the shape and extendingfrom points along the shape. The heart wall may be detected using theoperation discussed in FIGS. 7a-7 c, for instance. The shape which isassessed as having the closest fit to the border to be traced, by anaverage of the distance measurements, for instance, is chosen as theshape used in the continuation of the protocol.

The chosen shape is then fitted to the border to be traced by“stretching” the shape, in this example, to the endocardial wall. Thestretching is done by analyzing 48 lines of pixels evenly spaced aroundthe border and approximately normal to heart wall. The pixels along eachof the 48 lines are analyzed as shown in FIGS. 7a-7 c to find theadjacent endocardial wall and the chosen shape is stretched to fit theendocardial wall. The baseline between points 26 and 36 is not fit tothe shape but is left as a straight line, as this is the nominal planeof the mitral valve. When the shape has been fit to points along theheart wall, the border tracing is smoothed and displayed over the endsystole image as shown in the image 78 on the right side of the dualdisplay of FIG. 8. The display includes five control points shown as X'salong the border between the MMA landmark and the apex, and five controlpoints also shown as X's along the border between the apex landmark andthe LMA landmark. In this example the portion of line 48 between theapex and the mitral valve plane is also shown, as adjusted by thestretching operation.

With the end systole border drawn in this manner the ABD processor nowproceeds to determine the end diastole border. It does so, not byrepeating this operation on the end diastole image 16, but by finding aborder on each intervening image in sequence between end systole and enddiastole. In a given image sequence this may comprise 20-30 imageframes. Since this is the reverse of the sequence in which the imageswere acquired, there will only be incremental changes in the endocardialborder location from one image to the next. It is therefore to beexpected that there will be a relatively high correlation betweensuccessive images. Hence, the end systole border is used as the startinglocation to find the border for the previous image, the border thusfound for the previous image is used as the starting location to findthe border for the next previous image, and so forth. In a constructedembodiment this is done by saving a small portion of the end systoleimage around the MMA and the LMA and using this image portion as atemplate to correlate and match with the immediately previous image tofind the MMA and the LMA locations in the immediately previous image.The apex is located as before, by bisecting the angle between the upperportions of the septum and lateral LV wall, then locating theendocardium by the maximum slope of the brightness gradient. Since theLV is expanding when proceeding from systole to diastole, confidencemeasures include the displacement of the landmark points in an outwarddirection from frame to frame. When the three landmark points are foundin a frame, the appropriately scaled standard shape is fit to the threepoints. Another confidence measure is distention of the standard shapes;if a drawn LV border departs too far from a standard shape, the processis aborted.

Border delineation continues in this manner until the end diastole imageis processed and its endocardial border defined. The dual display thenappears as shown in FIG. 8, with endocardial borders drawn on both theend diastole and end systole images 76, 78.

As FIG. 8 shows, the endocardial borders of both the end diastole andend systole images have small boxes denoting the three major landmarksand control points marked by X's on the septal and lateral borders. Theclinician chooses the default number of control point which will bedisplayed initially; on the border 80 shown in FIG. 9 there are threecontrol points shown on the septal wall and four control points shown onthe lateral wall. The clinician can review the end diastole and systoleimages, as well as all of the intervening images of the loop if desired,and manually adjust the positions of the landmark boxes and controlpoint X's if it is seen that the automated process placed a border in anincorrect position. The clinician can slide a box or X along the borderto a new position, and can add more control points or delete controlpoints from the border. The process by which the clinician relocates abox or X laterally is known as rubberbanding. Suppose that the ABDprocessor had initially located the control point and border at theposition shown by circle 82 and dashed line 84, which the clinicianobserves is incorrect. The clinician can relocate the control pointlaterally by dragging the X with a screen pointing device to the newlocation as shown by 86. As the X is dragged, the border moves orstretches along with the X, thereby defining a new border as shown bythe solid line border 88. In this manner the clinician can manuallycorrect and adjust the borders drawn by the ABD processor. As theclinician laterally relocates a control point X, the ABD processorresponds by automatically recalculating the positions of the adjoiningborder and adjacent control points if necessary so that the borderremains smoothly continuous. The recalculation will not adjust theposition of a control point or landmark box which has been previouslymanually repositioned by the clinician, thereby preserving this expertinput into the border drawing process. If the clinician relocates alandmark box, the ABD processor recalculates and refits the entireborder to the landmarks and heart wall. Since the adjustment of oneborder in the image sequence can affect the borders of temporallyadjacent images in the sequence, the ABD processor will also respond toa manual adjustment by correlating the adjusted border with temporallyadjacent borders so that the manual adjustment is properly continuouslyrepresented in some or all of the images in the loop.

Another way to interactively adjust the drawn borders is to assembleonly the border tracings in a “stack” in time sequence from ED to ES orlater to form a surface defined by the borders which is viewed in threedimensions such as in a kinetic parallax display. The continuous surfaceformed by the borders can be assessed and adjusted as desired by arubberbanding technique know as active surface adjustment. If theclinician sees a point on the surface formed by the borders which is outof alignment with temporally adjacent tracings or the desired border,the clinician can pull or push on the surface with a pointing device.The active surface adjustment then conforms the adjacent borders and thesurface defined thereby to the adjustment made by the clinician, much asa balloon conforms when depressed at a point on its surface. Theclinician can thus observe the effect of an adjustment made to oneborder on the temporally surrounding borders of the cardiac cycle.

In a preferred embodiment the control points are not simply distributedat uniform intervals around the drawn border, but their positionscorrespond to constant anatomical locations from frame to frame over theheart cycle. This may be done by referencing the control points of theimage to those of a reference image through speckle tracking, featuretracking, or any kind of vector velocity or displacement processing.Since points in anatomy shown in an ultrasound image will exhibit asubstantially constant pattern of speckle from frame to frame, thecontrol points in other images can be located at points on theirrespective drawn borders which correspond to their characteristicspeckle locations on the reference image. When the control points arelocated at constant anatomical positions they will appear to move closertogether and then further apart through the heart cycle as the heartwall contracts and expands. When a control point X is relocated on aborder by the clinician, the corresponding control point X's on theother images are correspondingly relocated automatically to the newspeckle-tracked locations on each image. Such constant anatomicallocations for the control points are important when assessing localheart wall motion as discussed below.

Since each of the images shown in FIG. 8 is one image in the cardiacloop of images, the clinician can further verify the accuracy of theborders of the end diastole and end systole images 76, 78 by playing thecardiac loop of images behind the borders drawn on the display of FIG.8. This is done by selecting one of the images of FIG. 8, then selecting“Play” from the system menu to repetitively play the cardiac loop inreal time or at a selected frame rate of display behind the border. Inthe end diastole image 76 the endocardium is at its maximum expansion;hence, the endocardium in the loop should appear to move inward from andthen back to the endocardial border drawn on the end diastole image. Inthe end systole image 78 the endocardium is fully contracted; hence, theendocardium in the loop should appear to move outward and then back tothe border in this image. If the endocardium does not move in thismanner and, for example, is seen to pass through the border, a differentimage may need to be chosen for end diastole or end systole, or manualadjustment of a drawn border may be necessary. Of course, the loop andits drawn borders over the complete cardiac cycle can be replayed,enabling the clinician to view to endocardial tracing as it changes withthe heart motion in real time.

As the ABD processor is identifying the key landmarks and fittingborders to the sequence of images, it is periodically making confidencemeasurements to gauge the likelihood that the image borders are beingaccurately located and traced. For instance, if the septum is notclearly contrasted from the blood pool in the LV chamber, the automatedprocess will stop. If the various correlation coefficients do not exceedpredetermined thresholds the process will stop. Both spatial andtemporal confidence measurements are employed. For instance, if thecomputed border of an image varies too much from a standard shape ineither size or shape, the process will abort. This can arise if thelandmarks are located in unusual positions in relation to each other,for example. If the change in the computed border from one image in thesequence to another is too great, the process will likewise abort. Whenthe process stops, a message is displayed notifying the clinician of thereason for stopping the process, and gives the clinician the option tocontinue the automated process, to continue the automated process withor after clinician input, or for the clinician to acquire a new loop ofimages or manually trace the current images.

In the illustrated example of FIG. 8 the automatically drawn borders ofthe end diastole and end systole images are used to compute the heart'sejection fraction. This is done by an automatic modified Simpson's ruleprocess which divides the delineated heart chamber at each phase into astack of virtual disks. The diameter of each disk is used with the diskheight to compute an effective volume of each disk, and these volumesare summed to compute the heart chamber volume at both end diastole andend systole. The difference between the two yields the ejectionfraction, the volume or percentage of the heart volume which is expelledas pumped blood during each heart cycle. The ejection fractioncalculation is shown in the measurement box at the lower left handcorner of FIG. 8 and is constantly updated. Thus, if the clinicianshould adjust a drawn border by the rubberbanding technique, thecomputed volume of the heart during that phase will change, affectingthe ejection fraction calculation, and the new calculation immediatelyappears in the measurement box. As the clinician adjusts the drawnborders he instantaneously sees the effects of these changes on thecalculation of the ejection fraction.

In the previous example the clinician began by acquiring a cardiac loopon which to automatically trace borders. FIG. 10 shows an ultrasoundimage display in which a loop is acquired based upon the ability of theABD processor to automatically draw borders on the images. In theillustrated display the real time ultrasound image 10 is continuouslyviewed as in FIG. 1 as the clinician manipulates the transducer probe toacquire the desired four chamber view of the heart. As the clinicianmanipulates the probe the ABD processor is operative to attempt to drawborders on at least one of the images of each cardiac cycle. Using theR-wave timing of the ECG trace 12, the ultrasound system automaticallyselects the image or images to be traced from each loop. The selectedimage could be the first image of a cardiac cycle, the end diastoleimage, or the end systole image, for instance. As the ABD processorattempts to draw borders on the fly on the selected images of the realtime loops, the results of the ABD process for an image of each loop isshown as a small “thumbnail” image 92-98 below the real time image 10.In the illustrated example four thumbnail images are shown for fourconsecutive loops. Each time a new thumbnail is processed by the ABDprocessor it appears at the right side of the row of thumbnail images,the oldest thumbnail image disappears, and the row slides to the left.Initially the clinician may not be acquiring the heart in an orientationwhich is acceptable for the ABD process, at which time the progressionof thumbnail images will show no borders as the ABD processor is unableto successfully draw borders on the images. But as the clinicianmanipulates the probe to acquire the necessary viewing plane forsuccessful ABD performance and the images are acquired with betterclarity and definition, borders will appear on the progression ofthumbnail images as shown in the drawing figure. When the clinician isholding the probe at the necessary angulation relative to the heart sothat the ABD process is continuously successful, the progression ofthumbnail images will continuously show successfully drawn borders. Theclinician will then freeze the acquisition to capture one or more of thesuccessfully traced loops in the Cineloop memory, and will then selectone of the loops for full ABD processing and display as described above.Thus, the ABD processor is used to assist the clinician in manipulatingthe probe for successful image acquisition and in acquiring loops whichcan be successfully processed for border definition by the ABDprocessor.

Another way to indicate to the clinician that acceptable images for ABDprocessing are being acquired is by means of a graphical ABD successindicator. Such an indicator may be qualitative, quantitative, or both,as is the example shown in FIG. 10. At the right of the display of FIG.10 is a gauge 110 which is quantified from zero to 100%. When theclinician is acquiring images which are unsuitable for ABD processing,the gauge 110 is empty. But as suitable images begin to be acquired, acolor bar 112 begins to rise from the bottom of the gauge. Thequantization of the gauge indicates either the percentage of borderswhich were attempted and successfully drawn, or the changes in overallconfidence measures as discussed above. In the drawing a green bar is atthe 80% level, indicating that the ABD processor was able tosuccessfully process 80% of the images attempted over a recent intervalsuch as the last few heart cycles, or that the borders drawn achieved an80% confidence level of accuracy.

A third way to indicate ABD success to the clinician is to present drawnborders in real time on the real time images 10. The ABD processor canattempt to draw a border on a single image for each heart cycle, such asthe end systole image, and the successfully drawn border is displayedover the real time image for the duration of that heart cycle until thetime of the next end systole image. Alternatively, if sufficientprocessing speed is available, borders are calculated and displayed forevery image in the heart cycle. In either case, the drawn border willnot appear or will flicker on and off when unsuitable or marginalcardiac images are being acquired, but will constantly appear when asuccession of suitable images is being acquired, at which time theclinician aknows that the probe is oriented to acquire good four chamberviews for ABD processing.

In addition to the LV of four chamber views, the ABD processor of thepresent invention can also define borders in other types of ultrasoundimages. Short axis views can be processed for automatic borderdefinition, in which case the landmarks used can be the annulus or theoutflow track. Alternatively, the center of the heart chamber can befound from its contrast with the surrounding heart wall, then thedesired border located by radial extension and fitting of a circularstandard shape. The walls of blood vessels such as the carotid arterycan similarly be traced by identifying the center line of the vessel,then extending straight line shapes out from opposite sides of thecenter line to fit small line segments to the endothelial wall. Fetalanatomy such as the fetal cranium can also be automatically traced byuse of an elliptical shape.

With the ability to automatically draw borders of structures of theheart such as the endocardium on a complete loop of images, a number ofdiagnostic techniques become practical. For instance, FIG. 11illustrates a technique for assessing regional wall motion usingautomated border detection. The drawing of FIG. 11 represents anultrasound display in which the continuous motion of the endocardium ormyocardium is shown over several complete heart cycles. The ABDprocessor is operated as described above to draw a trace along theendocardial border or continuously through the myocardium of the imagesof one or more loops. The latter is performed by tracing the endocardialborder as described above, then drawing a curve parallel to and slightlylarger than the endocardial border curve. Such a curve will reliablypass continuously through the heart muscle. The border 100 for one suchimage is shown at the left side of the drawing, with the landmark pointsand control points numbered from one to eight in sequence around theborder. For analysis of wall motion the image points beneath the borderare Doppler processed to determine the velocity, Doppler power orvariance along the defined border. Thus, a tissue Doppler image line iscomputed along the endocardium or myocardium at locations defined by theautomatically drawn border. This Doppler processing is performed for thedefined border of each image in the loop or loops. The Doppler processedinformation from the moving tissue may be fundamental frequency signalsor harmonic frequency signals which can be processed as described inU.S. Pat. No. 6,036,643. The lines of Doppler values for all of theimages are displayed in straight vertical lines as shown at the rightside of FIG. 11 as indicated by the vertical sequence of numbers 1-8.The lines are arrayed sequentially adjacent to each other in the timesequence of the images. The Doppler values are preferably displayed incolor, thus forming a color M-mode display area 102. The display in area102 may be referred to as an ABD-TDI (ABD with tissue Doppler imaging)display. In the illustrated display the color Doppler lines for thefirst loop are arrayed across the area indicated by bracket L1, thecolor Doppler lines for the next loop are arrayed across the areaindicated by bracket L2, and the color Doppler lines for the third loopare arrayed across the area indicated by bracket L3, and so on. As thearrow at the bottom of the display area 102 indicates, the Doppler linesprogress in time in the horizontal direction. This display 102 thusshows in a continuum over the heart cycle the motion of the LVmyocardium. This display enables the clinician to follow the motion ofone point or region of the heart wall over a full cardiac cycle byobserving a horizontal row of the display. For instance, the heart wallat the apex of the heart is marked by 5 at the left of the area 102,corresponding to the apex landmark 5 on the border 100. By viewing theDoppler data (colors) to the right of 5 in area 102 the clinician isable to see the velocity or change in velocity or intensity of motion ofthe heart wall at the apex of the heart as it varies over the completeheart cycle or cycles. If a region of the wall is not moving due toinfarction or some other defect, it can be spotted by a change ordifference in color at a particular horizontal elevation in the ABD-TDIdisplay.

It will be appreciated that, since the LV heart wall is constantlyexpanding and contracting as the heart beats, the length of the line 100from the MMA, around the apex, and back to the LMA is constantlychanging in correspondence. If the control points are simply delineatedin even spacings around the line 100, they may not continuouslycorrespond to the same points of the heart wall through the full heartcycle. This is overcome by tracking the anatomy from a baseline ofcontrol points over the heart cycle, as by speckle tracking each localpoint of the heart wall along the ABD trace from frame to frame, asdescribed above. The different length lines are rescaled or normalizedto a common length so that a horizontal line extended to the right fromeach number at the left of the display 102 will relate to the same pointor region of the heart wall over the continuum of tissue Doppler lines.

An ABD-TDI display may also be formed from short axis images of theheart. In short axis views the heart wall exhibits a ring shape. Asdescribed previously, the endocardium can be traced automatically foreach frame of the cardiac cycle and a parallel, slightly larger circlethan the tracing can be drawn through the myocardial muscle in theimages. Doppler values are acquired around each of these circles, whichare displayed in a continuum of lines in the format shown in area 102 ofFIG. 11. Thus, the display format 102 may be used for either short orlong axis views of the heart.

Another application for automatically drawn cardiac borders is shown inFIG. 12. In this illustration the border 300 represents the endocardialborder defined by automated border detection as described above, with aline 306 for the mitral valve plane at the bottom. A second, slightlylarger border 302 is drawn around the first border 300. This secondborder may be an ABD-produced border of the epicardium, or it may be atrace spaced by a predetermined lateral distance d normal to theendocardial border 300. In this latter case, the trace 302 can passcontinuously through the myocardium. Thus, Doppler values along thetrace 302 would yield motional measures taken in a central portion ofthe heart muscle. The space between the two traces can be divided intosmall areas 304 and the Doppler values within each area integrated toproduce a measure of regional wall motion at a particular location onthe LV wall. These measures are made using ABD processing of many or allof the images of the cardiac loop to quickly and accurately providequantified measures of cardiac performance over most or all of thecardiac cycle.

The measurements made from the areas 304 can be used to automaticallyfill out an anatomically corresponding scorecard for cardiacperformance. For example, FIG. 13a shows a graphical representation 310of the LV in a 4-chamber view, with the myocardium divided into numberedareas. The region numbered 6 on the anatomical scorecard 310 correspondsto the small areas 304 a-304 d which were defined by automatically drawnborders. The measurements taken in these areas 304 a-304 d can beaggregated and used to automatically place a score on the scorecard 310for region 6, which may be numerical or qualitative, for example, acoded color. The score can be a peak or average value measured for onephase of the heart cycle or taken over all the frames of the full heartcycle. FIG. 13b illustrates a similar anatomical scorecard 312 for ashort axis view of the heart, which may be used to score images withautomatically drawn borders acquired from that view. A scorecard may befilled in for just a single image frame, for a group of image framestaken together, or a scorecard may be completed for each frame of acardiac sequence. In the latter case, color coded scorecards can beplayed in rapid succession in a real time (or slower or faster) loop ofimages of the scorecards, enabling the clinician to view the timevariation of a region of the heart in a segment of the scorecard whichis stationary on the display screen from frame to frame.

Automatically drawn cardiac borders may also be used to define themyocardial area in contrast-enhanced images or loops. The addition of acontrast agent to the cardiac imaging exam allows the physician toassess how well the heart muscle is being perfused with blood.Automatically computed borders may be used as input into a variety ofperfusion quantification algorithms. Automatically drawn cardiac bordersand perfusion information presented simultaneously in an image or loopis a powerful combination since the clinician can assess wall motion,thickening, and perfusion simultaneously. Given that the borders areknown, the thickness of the myocardial walls between the endocardial andepicardial edges can be determined on a segment-by-segment basis asshown in FIG. 12. Perfusion information quantified by an independentalgorithm may also be displayed side-by-side with the quantitative wallthickening information. Quantitative perfusion information and wallthickening nay also be parametrically combined and presented on asegment-by-segment basis in a color coded display for Doppler and wallmotion integration.

Another diagnostic technique made practical by automatic borderdetection is strain rate analysis of cardiac performance. The strainrate is a measurement computed as the axial derivative of the velocityof the tissue, and can lead to a representation of the relativedeformation of the tissue during contraction or expansion. Theconventional way to compute strain rate in an ultrasound image is tofind Doppler velocity values along the ultrasound beams, then to computethe spatial gradient as a derivative using successive velocity valuesalong the beam. This spatial gradient of velocity is thus stronglydependent upon the variable relationship between the beam directions andthe anatomy in the image, which means that the strain rate values canchange as the probe is moved. The present inventors prefer to use astrain rate calculation which is dependent upon the direction of tissuemotion rather than an arbitrary beam direction. Accordingly the presentinventors calculate strain rate in the direction of the velocity vectorof tissue motion. In order to do this it is necessary to have not onlyvelocity values for the tissue pixels in an image but also the directionor vectorial component of the motion, which can be obtained by knownvector Doppler techniques. The differential between adjacent pixels inthe direction of motion is then computed as the strain rate. The strainrate can be computed from fundamental frequency echo information, orfrom harmonic frequency signals which can be more clutter-free than thefundamental frequency signals.

FIG. 14a shows two traces which have been automatically drawn over theborders of a 4-chamber view of the LV. The border 250 is drawn to definethe endocardium and the border 252 has been drawn to define theepicardium of the LV. A third trace 260 is automatically drawn betweenthe endocardial and epicardial borders. This third trace 260 willreliably pass continuously through the myocardium. These traces enablethe strain rate to be computed for the two major components of motion ofthe LV. One of these components is the contraction and expansion ofadjoining cells in the heart muscle. This motion is generally along thedirection of the trace 260. A strain rate representation of thiscellular motion can be found by differentiating the velocity values ofsuccessive points A-A′ along trace 260 as shown in the drawing. Theoverall motion of the heart chamber as the muscle cells contract andexpand is toward and away from the center of the heart chamber. A strainrate representation of this second motional component is computed bydifferentiating velocities in a direction normal to the drawn borderssuch as at points B-B′across the heart muscle. The strain rate socalculated along the myocardium is preferably displayed in a colorcodedrepresentation. A similar set of strain rate measurements can be madeusing borders 270 (endocardium) 272 (epicardium) and trace 280(myocardium) drawn on short axis views of the heart such as that shownin FIG. 14b. In that drawing muscle cell contraction and expansion isused to compute strain rate in the circumferential direction such aswould be computed from the velocities at points A-A′ in the image.Radial components of expansion and contraction are represented in astrain rate display by differentiating in the radial direction such asby using the velocities at points B-B′, C-C′, and D-D′. The strain rateover the full cardiac cycle can be displayed by computing the strainrate around the entire border for each frame in the cardiac cycle, thendisplaying the strain for each frame as a vertical line in a timesequence of lines as shown by display 102 in FIG. 11.

FIGS. 15a and 15 b illustrate the use of automated border detection inthree dimensional imaging. The previous examples have shown how borderscan be automatically drawn on two dimensional cardiac images. Thetechnique described above is effective for defining the borders of threedimensional cardiac images also. If a three dimensional cardiac image isproduced by acquiring a series of spatially adjacent 2D image planes ofthe heart, the ABD process described above can be performed on eachcomponent frame to define a series of boundaries which together define asurface of the heart such as the endocardial surface. If the threedimensional cardiac image is produced from ultrasonic beams steered inthree dimensions to scan the heart three dimensionally in real time asdescribed in U.S. patent application Ser. No. 09/645,872 entitled“ULTRASONIC DIAGNOSTIC IMAGING OF THE CORONARY ARTERIES,” the resultingthree dimensional data set can be divided into a series of parallelplanes which are processed as described above to define a series ofplanar borders which can be assembled to provide a boundary such as theheart wall. Preferably the three dimensional data set is processed threedimensionally, in which case advantage is taken of the contiguous natureof the heart wall in three dimensions in the data set to more reliablydefine the three dimensional border In either case, the resultant threedimensional border of the LV endocardium may appear as shown in FIG.15a, a somewhat elongated pouch-like surface which is closed at the apexend A and open at the mitral valve plane A′. FIG. 15a represents thethree dimensional endocardial surface traced at one phase of the heartcycle. In each 3D image of a 3D cardiac loop the endocardial surfacewill be slightly different as the LV continuously contracts and expandsduring the heart cycle. Thus a different border surface 200 can becomputed for each three dimensional image in the loop. Since the speedof sound may not enable the ultrasonic data for a full 3D image to beacquired at the desired 3D frame rate, the 3D images may be built upover time by using a heart gate triggered from the ECG waveform toacquire the data for a portion of a 3D image at specific phases of theheart over numerous cardiac cycles until the full data set necessary toproduce 3D images of the desired temporal spacing over the entire heartcycle is acquired.

The 3D image of the endocardium in FIG. 15a can be produced by Dopplerprocessing, thereby revealing the velocity, variance, or Doppler powerat each point on the LV wall by rotating and examining the endocardialtissue Doppler surface 200. Another way to view the Doppler informationfor the entire endocardium is to “unwrap” the tissue Doppler surface 200to a two dimensional form as shown in FIG. 15b. In this illustration theapex is located at A and the mitral valve plane extends along the bottomedge between A′ and A′. In this display the clinician can examine themotion of the entire endocardium in one view. Such a display showsmotion at only one phase of the heart cycle, the phase indicated bycursor 14 below the ECG waveform 12 of the display, and thus it isdesirable to unwrap all of the endocardial surfaces from all of the 3Dimages of the heart cycle and arrange them in a “stack” so that theclinician can view them sequentially in any order. If the clinicianspots a motional abnormality on one of the unwrapped images, such as inthe area denoted by box 202, she can focus in on this cardiac walllocation where the abnormality is seen. She can then scan through thestack of images in the box 202 in either temporal order to examine theabnormality in greater detail over the full heart cycle. Alternately,the clinician can draw a line through the abnormality in box 202, thendisplay the tissue Doppler values along that line from all the images inthe sequence in an ABD-TDI display 102 as described above.

If real time 3D imaging capability is not available, a 3D diagnosis canstill be performed by acquiring multiple image planes of a chamber ofthe heart at different orientations, which are then processed byautomatic border detection. An ultrasound system which acquiresultrasound information from only selected planes of an organ of the bodyis described in U.S. patent application Ser. No. 09/641,306, entitled“METHOD OF CREATING MULTIPLANAR ULTRASONIC IMAGES OF A THREE DIMENSIONALOBJECT.” FIG. 15c is an illustration of the endocardial border 200viewed from the apex A in the center of the drawing, which is how theheart may be viewed by a transducer probe placed for an apical view asdescribed above. With the probe so located, ultrasound information isacquired from three planes which pass through the heart chamber, labeled204, 206 and 208 in the drawing. In this drawing the planes are viewededge-on, and in this example the three planes intersect in the vicinityof the apex of the LV. The ultrasound information from the three planesmay be acquired at a particular phase of the heart cycle chosen by andECG heart gate, or over the full cardiac cycle which may also beassisted by ECG gated acquisition over a number of cardiac cycles. TheLV endocardial borders in the images of the three planes areautomatically drawn as described above and analyzed.

A quick method for identifying a region of the heart where more detailedstudy is required is to score cardiac performance on a symbolicrepresentation of the heart. One such symbolic representation is thebullet scorecard 210 shown in FIG. 15d. The scorecard 210 represents theheart muscle of a chamber of the heart as if the myocardium were spreadout in a single plane with the apex at the center of the scorecard andthe juncture of the myocardium and the mitral valve plane located aroundthe perimeter of the scorecard. Each sector of the scorecard 210extending from the center to the perimeter represents a differentsection of the heart muscle extending from the apex to the mitral valveplane. The areas in the scorecard are numbered to refer to specificareas of the heart wall. For instance the image plane 204 of FIG. 15cwould intersect areas 1, 7, the center of the scorecard, and areas 10and 4. The image plane 206 of FIG. 15c would intersect areas 6, 12, 16,14, 9 and 3 of the scorecard, and the image plane 208 of FIG. 15c wouldintersect areas 5, 11, 15, 13, 8 and 2 of the scorecard. TheDoppler-detected motion on an automatically drawn border in one or moreimage frames in the image planes is used to enter data in the scorecard210. The scorecard is filled in automatically using the motioninformation from the automatically drawn borders to indicate areas ofthe heart where detailed diagnosis is warranted. For instance, ifcardiac behavior in the plane 204 of the LV is normal, areas 1, 7, 10,and 4 can be displayed in green on the ultrasound system display. If anunusual characteristic such as abnormal motion is sensed in the vicinityof the juncture of the myocardium and the mitral valve plane, area 1 maybe displayed in yellow (for mild irregularity) or red (for a seriousirregularity), to caution the clinician to look more closely at thisarea. Numerical scores may be used in addition to or alternatively tocolor-coding. A preferred four-tiered scoring system for cardiacperformance is to score regions of the heart muscle as being eithernormal, hypokinetic, dyskinetic, or akinetic. Thus the displayed bulletscorecard with its color-coded or numerically scored areas will pointthe clinician to regions of the heart where more detailed diagnosisshould be performed.

It is preferable, of course, to use a complete 3D data set to fill inthe scorecard 210. For instance, the defined heart wall 200 of FIG. 15acan be “flattened,” and spread in a circle about the apex so that eacharea of the myocardium in the data set corresponds to an area of thescorecard. The motional data over a section of the flattened heart wall200 can be averaged to fill in a corresponding section of the bulletscorecard 210. For example, the motional data over section 212 of theendocardial data 200 can be averaged to automatically compute a score(either quantitative or qualitative) for corresponding area 5 of thescorecard 210. The scores for section 212 from a plurality ofendocardial data sets taken over the full heart cycle can also beaveraged or outlying values detected to produce a scorecard of averagesof cardiac performance or greatest degrees of abnormal performance.

FIG. 16 illustrates an ultrasound system constructed in accordance withthe present invention. A probe or scanhead 410 which includes a 1D or 2Darray transducer 412 transmits ultrasonic waves and received ultrasonicecho signals. This transmission and reception is performed under controlof a beamformer 420 which processes in received echo signals to formcoherent beams of echo signals from the anatomy being scanned. The echoinformation is Doppler processed by a Doppler processor 430 when ABD-TDIinformation or strain rate information is to be presented, and theprocessed Doppler information is coupled to an image processor 440 whichforms 2D or 3D grayscale or Doppler images. The images pass through aCineloop memory 460 from which they may be coupled directly to a videoprocessor 470 for display on an image display 480. The images may alsobe applied to an ABD processor which operates on the 2D or 3D images asdescribed above to define the anatomical borders and boundaries in theimages. The defined borders are overlaid on the images which are coupledto the video processor 470 for display. The system may operate to defineand display borders on loops of images saved in the Cineloop memory 460,or to display borders drawn on real time images produced during livescanning of a patient.

What is claimed is:
 1. A method for displaying information derived byautomatic border detection in cardiac ultrasound images comprising:acquiring a sequence of cardiac ultrasound images; automatically tracinga border of moving tissue in each of the images; computing motion alongeach of the traced borders; and displaying the computed motion along thetraced borders as a function of time in one direction and tissuelocation in another direction.
 2. The method of claim 1, wherein theborder tracings are nonlinear, and wherein the computed motion alongeach traced border is displayed linearly.
 3. The method of claim 1,wherein the computed motion is displayed in colors corresponding tovelocity.
 4. The method of claim 1, wherein the border traced is theendocardium.
 5. The method of claim 1, wherein the border traced is theepicardium.
 6. The method of claim 1, wherein the tracing is offset fromthe endocardium in the direction of the epicardium.
 7. The method ofclaim 1, further comprising: automatically tracking correspondinganatomical points on each of the borders from image to image; anddisplaying the motion of the corresponding anatomical points in spatialcorrespondence on a display.
 8. The method of claim 7, whereinautomatically racking comprises tracking the speckle patterns ofanatomical points along borders of the images.
 9. The method of claim 7,wherein automatically tracking comprises tracking the tissue patterns ofanatomical points along borders of the images.
 10. The method of claim1, further comprising manually adjusting one or more of theautomatically traced borders.
 11. The method of claim 1, wherein time isshown along one axis and location on the border is shown along the otheraxis.
 12. A method for analyzing cardiac performance by automatic borderdetection in cardiac ultrasound images comprising: acquiring a sequenceof cardiac ultrasound images; automatically tracing a border of theheart in one or more images; segmenting the myocardium along each of thetraced borders; computing scores for the myocardial segments; andautomatically entering the scores on a scorecard having entries relatingto the myocardial segments, wherein the scorecard exhibits an anatomicalcorrespondence to the myocardium.
 13. The method of claim 12, whereinthe entries comprise alphanumeric entries.
 14. The method of claim 12,wherein the entries comprise colored or shaded entries.
 15. The methodof claim 12, wherein acquiring comprises acquiring a three dimensionalimage of the heart; and wherein automatically tracing comprisesautomatically delineating the heart wall in a three dimensional image.16. The method of claim 15, wherein analyzing comprises quantifying wallmotion for myocardial segments of the three dimensional image, whereinquantified wall motion is displayed in the three dimensional image bycolors indicating wall motion.
 17. The method of claim 15, whereinanalyzing comprises computing scores for myocardial segments of thethree dimensional image; and further comprising: automatically enteringscores on a scorecard having entries relating to the myocardialsegments.
 18. The method of claim 17, wherein the scorecard comprises anelectronic scorecard having areas corresponding to areas of the heartwall delineated in the three dimensional image.
 19. A method foranalyzing cardiac performance by automatic border detection in cardiacultrasound images comprising: acquiring two or more images in differentimage planes which intersect a chamber of the heart; automaticallytracing a border of the heart in each of the images; and analyzingcardiac performance in regions of the heart delineated by theautomatically traced borders.
 20. The method of claim 19, whereinacquiring further comprises acquiring ultrasonic signals by gatedacquisition.
 21. The method of claim 19, wherein acquiring furthercomprises acquiring ultrasonic signals by electronic beam steering. 22.The method of claim 20, wherein acquiring further comprises acquiringultrasonic signals by mechanical beam steering.
 23. The method of claim19, wherein analyzing comprises computing and displaying regional wallmotion along each of the automatically traced borders.
 24. The method ofclaim 19, wherein analyzing comprises computing scores for delineatedregions of the heart wall; and further comprising: automaticallyentering scores on a scorecard having entries relating to the heart wallregions.
 25. The method of claim 24, wherein the scorecard comprises anelectronic scorecard having entries corresponding to regions of theheart wall.
 26. A method for displaying measures of cardiac performanceby automatic border detection in cardiac ultrasound images comprising:acquiring three dimensional ultrasonic motional data of a chamber of theheart; automatically tracing a border of the heart in the threedimensional ultrasonic data; and displaying in three dimensions motionaldata of a region of the heart delineated by the border.
 27. The methodof claim 23, wherein the region of the heart comprises the heart wall.28. The method of claim 26, wherein displaying further comprisesdisplaying in two dimensions the region of the heart delineated by theborder.
 29. The method of claim 28, wherein displaying further comprisesdisplaying in two dimensions motional ultrasonic information of theregion of the heart delineated by the border.
 30. The method of claim26, further comprising delineating a region of interest on the displayedregion of the heart.
 31. A method for displaying information derived byautomatic border detection in cardiac ultrasound images comprising:acquiring a sequence of cardiac ultrasound images; automatically tracinga border of moving tissue in each of the images; computing motion alongeach of the traced borders utilizing harmonic signal informationcorresponding to the traced borders; and displaying the computed motionalong the traced borders as a function of time in one direction andtissue location in another direction.
 32. The method of claim 31,wherein computing comprises computing motion along the traced borders byDoppler processing harmonic signal information from moving tissue.
 33. Amethod for analyzing cardiac performance by automatic border detectionin cardiac ultrasound images comprising: acquiring a sequence of cardiacultrasound images; automatically tracing a border of the heart in thecardiac images; segmenting the myocardium along each of the tracedborders; computing scores for the myocardial segments; and automaticallyentering the scores on scorecards having entries anatomically relatingto the myocardial segments.
 34. The method of claim 33, furthercomprising displaying the scorecards on a display in rapid timesuccession.
 35. The method of claim 34, wherein rapid time succession isone of faster than, slower than, or equal to the real time display rateof the cardiac ultrasound images.